Stochastic Zeroth-Order Functional Constrained Optimization: Oracle Complexity and Applications

Functionally constrained stochastic optimization problems, where neither the objective function nor constraint functions are analytically available, arise frequently in machine learning applications. In this work, assuming we only have access to noisy evaluations of and functions, propose analyze zeroth-order algorithms for solving class problem. When domain is [Formula: see text], there m establish oracle complexities order text] convex nonconvex settings, respectively, ϵ represents accuracy solutions required appropriately defined metrics. The established are, our knowledge, first such results literature functionally problems. We demonstrate applicability by illustrating their superior performance on problem hyperparameter tuning sampling neural network training. Funding: K. Balasubramanian was partially supported a seed grant from Center Data Science Artificial Intelligence Research, University California–Davis, National Foundation [Grant DMS-2053918].